Projector-camera Systems for Immersive Training

Real time computer graphics are limited in that they can only be displayed on projection screens and monitors. Monitors and projection screens cannot be used in live fire training or scenarios in which the displays could be physically damaged by trainees. To address this issue, we have developed projection systems using computer vision based color correction and image processing to project onto non-ideal surfaces such as painted walls, cinder blocks, and concrete floors. These projector-camera systems effectively paint the real world with digital light. Any surface can become an interactive projection screen allowing unprepared spaces to be transformed into an immersive environment. Virtual bullet holes, charring, and cracks can be added to real doors, walls, tables, chairs, cabinets, and windows. Distortion correction algorithms allow positioning of projection devices out of the field of view of trainees and their weapons. This paper describes our motivation and approach for implementing projector-camera systems for use within the FlatWorld wide area mixed reality system.